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DETAILED ACTION 

1. This action is responsive to communications: Amendment filed 1/3/2008. 

2. Claims 19-22, 24-30, 32-35 are pending in the case. Claims 19 and 25 are independent 
claims. 

Claim Rejections - 35 USC § 112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

4. Claims 34 and 35 are rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in the relevant 
art that the inventor(s), at the time the application was filed, had possession of the claimed 
invention. While elements of the claimed subject matter appear throughout the specification, 
assets as described in claim 19 appear to be inconsistent with the assets described in claim 34. 
There also does not appear to be any support for code generation. Applicant has not pointed out 
where in the application as filed this claim has support, nor is it apparent. 

5. Claims 34 and 35 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

As described above, claim 34 appears to contradict both the specification and the 
independent claim. As such the scope of the claim cannot be determined. For examining 
purposes only, claim 34 will be rejected under the same rationale as its parent. 
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Claim Rejections - 35 USC §102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1(2) of such treaty in the English language. 

7. Claims 19, 20, 22, 24-26, 28, 30 and 32-34 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Bergman (US006564263B1, 5/13/2003). 

Regarding independent claim(s) 19, Bergman teaches: 

checking whether an asset for display (multimedia content) is comprised of one or more 
photo data and one or more audio data (col. 19, 11. 54-58, checks to see what the content is 
comprised, which in the normal course operation includes photo, image col. 8, 11. 28-36, and 
audio, col. 8, 11. 37-39); 

extracting reference information needed for displaying the photo data and the one or 
more audio data from the asset; (description scheme, col. 19, 11. 54-55); 

displaying the photo data and the one or more audio data, using the extracted reference 
information, based on a meta data including time information indicating time during which each 
piece of the photo data is displayed while the audio data is being provided (multimedia is 
displayed, col. 7, line 59, the multimedia is generated/synthesized according to (based on) intra- 
object relationships, col. 20, 11. 34-47, which includes a position and a time when the photo data 
is displayed (col. 6, 11. 53-56).) 
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Regarding independent claim(s) 25, Bergman teaches: 

displaying the asset using the generated metadata (multimedia is displayed, col. 7, line 
59, the multimedia is generated/synthesized according to (based on) intra-object relationships or 
meta-data, col. 20, 11. 34-47) 

the asset is comprised of audio, (col. 8, 11. 37-39) and one or more photo data (image, col. 
8, 11. 28-36) and is displayed by use of a asset corresponding to the reference information, 
wherein, the reproduction information comprises information on a time when the text data is 
displayed (metadata includes a time when the text data is displayed, col. 6, 11. 53-56). Inherently, 
this metadata must have been generated. 

Regarding dependent claim(s) 20, 26, Bergman teaches a markup language (col. 14, 11. 3-29). 
Regarding dependent claim(s) 22, 30, Bergman teaches identification information to identify 
the asset (annotations, col. 8, 11. 55-58). 

Regarding dependent claim(s) 24, 32, Bergman teaches information on an attribute of an asset 
(annotations, col. 8, 11. 55-58). 

Regarding dependent claim(s) 28, Bergman teaches the elements are defined by a schema 
(scheme, col. 18,11. 63-67). 

Regarding dependent claim(s) 33, Bergman teaches an instance of a class that comprises the 
audio data and still photo data ("Infopyramid", para. col. 8, 11. 10-19). 
Regarding dependent claim(s) 34, as described above, this claim is rejected under the same 
rationale as its parent. 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

10. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bergman. 
Regarding dependent claim(s) 29, Bergman does not give the elements the exact names in the 
claims. However, the differences in the claimed invention are only found in the nonfunctional 
descriptive material and are not functionally involved in the steps recited. All the steps of the 
function would be performed the same way regardless of whether or not the name of the element 
is ' Audio WithStill" or anything else (for example "Audlmg"). Thus, this descriptive material 
will not distinguish the claimed invention from the prior art in terms of patentability, see In re 
Gulack, 703 F.2d 1381, 1385, 217 USPQ 401, 404 (Fed. Cir. 1983); In re Lowry, 32 F.3d 1579, 
32 USPQ2d 103 1 (Fed. Cir. 1994). Therefore, it would have been obvious to a person of 
ordinary skill in the art at the time the invention was made to call the elements any name 



Application/Control Number: 10/559,374 Page 6 

Art Unit: 2178 

(including ' Audio WithStill") because such data does not functionally relate to the steps in the 
method claimed and because the subjective interpretation of the data does not patentably 
distinguish the claimed invention. 

11. Claims 21 and 27are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bergman as applied to claims 18 and 25above, and further in view of Applicant's Admitted 
Prior Art. 

Regarding dependent claim(s) 21, 27, Bergman teaches the meta-data is described according to 
MPEG-7, a known standard (col. 14, 11. 4-8). Bergman uses this standard rather then MPV. 
Applicant admits that MPV was a known standard (para. 3). It would have been obvious to one 
of ordinary skill in the art at the time of the invention to substitute the MPV standard for the 
MPEG-7 standard because the substitution of one known element for another would have yielded 
predictable results (multimedia is still displayed according to the meta-data) to one of ordinary 
skill in the art. 

12. Claim 35 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bergman as 
applied to claim 34 above, and further in view of Alvesalo, (US 20030222899 Al, 
12/04/2003). 

Regarding dependent claim(s) 35, Bergman does not explicitly disclose transitions. Alvesalo 
teaches defining the type of transition between assets (para. 27). It would have been obvious to 
one of ordinary skill in the art at the time of the invention to combine Alvesalo and Bergman to 
take advantage of advanced capabilities of hardware (Alvesalo, para. 27). 

Double Patenting 

13. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 



Application/Control Number: 10/559,374 
Art Unit: 2178 



Page 7 



improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection 
is appropriate where the conflicting claims are not identical, but at least one examined 
application claim is not patentably distinct from the reference claim(s) because the examined 
application claim is either anticipated by, or would have been obvious over, the reference 
claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re 
Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re 
Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may 
be used to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent either is shown to be commonly owned 
with this application, or claims an invention made as a result of activities undertaken within the 
scope of a joint research agreement. 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 
3.73(b). 

14. Claims 19-22, 24-30, and 32 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claims 6-10 and 12-18 of 
copending Application No. 11/415,096. Although the conflicting claims are not identical, 
they are not patentably distinct from each other because: 

Regarding independent claim(s) 19, every element of the claim is anticipated by claim 9 of the 
co-pending application (which incorporates its parent, claim 5), except the co-pending claim is 
the method that carried out by the apparatus of the instant claim. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to use a computer (including a memory 
under control of a processor) to carry out the method of the co-pending claim, because a 
computer was one of several predictable finite solutions for implementing the method. 

The remainder of the instant application's claims corresponds to the co-pending claims 
according to the table below, and differs from but is obvious from those claims for the same 



rationale as claim 19. 
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Instant 


19 


20 


21 


22 


24 


25 


26 


27 


28 


29 


30 


32 


Co-pending 


9 


6 


7 
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10 


17 


12 


13 


14 


15 


16 


18 



This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 

15. Claims 19-22, 24-30, and 32 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claims 16-20 and 24-30 of 
copending Application No. 10/948,316. Although the conflicting claims are not identical, 
they are not patentably distinct from each other because: 

Regarding independent claim(s) 19, every clement of the claim is anticipated by claim 19 of 
the co-pending application (which incorporates its parent, claim 15), except the co-pending 
claims teach videos displayed in relation to audio, while the instant application teaches photos 
displayed in relation to audio. It would have been obvious to one of ordinary skill in the art at 
the time of the invention to display photos in relation to audio instead of video, as they were both 
a form of multimedia intended to be displayed in conjunction with other multimedia, and would 
have received the same benefit by displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 


19 


20 


21 


22 


24 


25 


26 


27 


28 


29 


30 


32 


Co-pending 


19 


16 


17 


18 


20 


29 


24 


25 


26 


27 


28 


30 



This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 
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16. Claims 19-22, 24-30, and 32 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claims 28, 30-34, 36-41 of 
copending Application No. 10/949,474. Although the conflicting claims are not identical, 
they are not patentably distinct from each other because: 

Regarding independent claim(s) 19, every element of the claim is anticipated by claim 28 of 
the co-pending application except the co-pending claims teach text displayed in relation to any 
multimedia, while the instant application teaches photos displayed in relation to audio. It would 
have been obvious to one of ordinary skill in the art at the time of the invention to display photos 
instead of text, in relation to audio instead of any multimedia, as they were all forms of 
multimedia intended to be displayed in conjunction with other multimedia, and would have 
received the same benefit by displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 


19 


20 


21 


22 


24 


25 


26 


27 


28 


29 


30 


32 


Co-pending 


28 


30 


31 


32 


33 


34 


36 


37 


38 


39 


40 


41 



This is a provisional obviousness-type double patenting rejection because the conflicting claims 
have not in fact been patented. 

17. Claims 19-22, 24-30, and 32 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claims 18-22, and 25-31of 
copending Application No. 10/949,253. Although the conflicting claims are not identical, 
they are not patentably distinct from each other because: 
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Regarding independent claim(s) 19, every element of the claim is anticipated by claim 28 of 
the co-pending application except the co-pending claims teach photo displayed in relation to any 
video, while the instant application teaches photos displayed in relation to audio. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to display photos in 
relation to audio instead of any video, as they were all forms of multimedia intended to be 
displayed in conjunction with other multimedia, and would have received the same benefit by 
displaying them according to a temporal relationship. 

The instant application's claims below correspond to the co-pending claims according to 
the table below, and differ from but are obvious from those claims for the same rationale as 
claim 19. 



Instant 


19 


20 


21 


22 


24 


25 


26 


27 


28 


29 


30 


32 


Co-pending 


21 


18 


19 


20 


22 


30 


25 


26 


27 


28 


29 


31 



This is a provisional obviousness-type double patenting rejection because the conflicting claims 
have not in fact been patented. 

Response to Arguments 

18. Applicant's arguments filed 1/3/2008 have been fully considered but they are not 
persuasive. 

Applicant alleges that Bergman does not specifically disclose an object having audio and 
photo data. Bergman teaches that objects are contemplated as being several different modalities, 
such as text, video, image (photo) and audio. Bergman teaches that that among the options are 
the claimed audio and image (photo). Therefore Bergman teaches an option that the objects 
described as having a spatial-temporal relation can be audio and image. Optional inclusion 
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teaches both embodiments that do and do not include the options. See MPEP 2123 I. As such, 
the limitation is anticipated. 



Conclusion 

19. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ADAM M. QUELER whose telephone number is (571)272-4140. 
The examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on (571) 272-4124. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Stephen S. Hong/ 

Supervisory Patent Examiner, Art Unit 
2178 
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